Homogeneity of Effects: Interaction

Many medical studies are carried out to investigate the size of some effect, for example the treatment effect in a controlled clinical trial or the risk of exposure to some substance in an observational study. To this end it is usual to compare groups of subjects, such as those receiving active treatment with those getting placebo or those exposed with those not exposed. The comparison is often made using a statistical technique, such as a t-test, a χ test, ANCOVA. If these techniques (or indeed many other statistical techniques) are applied to the groups as a whole then only one estimate of the size of an effect will be obtained, i.e. there is an implicit assumption that the effect size is constant across different types of patients or subjects. In practice this may not be sensible, for example women may respond to a hormonal treatment differently from men or the effect of exposure to an environmental toxin may be different for smokers and non-smokers. Such heterogeneity of effect is usually investigated by repeating analyses on selected subgroups. Virtually any trial report will include a series of subgroup analyses, and manoeuvres with similar motivation occur in many epidemiological studies. Although the biology or medicine behind this approach may be sensible, there are several statistical traps for the unwary and some of these are discussed in this note.